When Do Standard Approaches for Measuring Vocabulary Difficulty, Syntactic Complexity and Referential Cohesion Yield Biased Estimates of Text Difficulty?

Many widely-used approaches for assessing text difficulty provide a single prediction equation that is assumed to hold for texts belonging to a variety of different genres. This paper demonstrates that such models tend to overpredict the difficulty of informational texts while simultaneously underpredicting the difficulty of literary texts. Mechanisms that may account for these effects are examined. Results suggest that the estimated effects can be traced to certain frequently used measures of vocabulary difficulty, syntactic complexity and referential cohesion.